"""
多级缓存系统
实现内存缓存、Redis缓存和帧缓冲区的多级缓存架构
支持实时视频流处理的高性能缓存需求
"""
import asyncio
import logging
import time
import json
import pickle
import hashlib
from datetime import datetime, timedelta
from typing import Dict, List, Optional, Any, Union, Tuple
from collections import OrderedDict, deque
from dataclasses import dataclass, asdict
import numpy as np
from enum import Enum

try:
    import redis.asyncio as redis
    REDIS_AVAILABLE = True
except ImportError:
    REDIS_AVAILABLE = False
    logging.warning("Redis not available. Multi-level cache will use memory only.")

from core.config import get_settings

logger = logging.getLogger(__name__)
settings = get_settings()


class CacheLevel(Enum):
    """缓存级别枚举"""
    MEMORY = "memory"
    REDIS = "redis"
    FRAME_BUFFER = "frame_buffer"


@dataclass
class CacheEntry:
    """缓存条目"""
    key: str
    value: Any
    created_at: datetime
    expires_at: Optional[datetime]
    access_count: int = 0
    last_accessed: Optional[datetime] = None
    size_bytes: int = 0
    cache_level: CacheLevel = CacheLevel.MEMORY


@dataclass
class CacheStats:
    """缓存统计信息"""
    total_requests: int = 0
    cache_hits: int = 0
    cache_misses: int = 0
    memory_hits: int = 0
    redis_hits: int = 0
    frame_buffer_hits: int = 0
    evictions: int = 0
    total_size_bytes: int = 0
    avg_access_time_ms: float = 0.0
    
    @property
    def hit_rate(self) -> float:
        """缓存命中率"""
        if self.total_requests == 0:
            return 0.0
        return self.cache_hits / self.total_requests
    
    @property
    def memory_hit_rate(self) -> float:
        """内存缓存命中率"""
        if self.total_requests == 0:
            return 0.0
        return self.memory_hits / self.total_requests


class LRUCache:
    """LRU内存缓存实现"""
    
    def __init__(self, max_size: int = 1000, max_memory_mb: int = 512):
        self.max_size = max_size
        self.max_memory_bytes = max_memory_mb * 1024 * 1024
        self.cache: OrderedDict[str, CacheEntry] = OrderedDict()
        self.current_memory_bytes = 0
        self._lock = asyncio.Lock()
    
    async def get(self, key: str) -> Optional[CacheEntry]:
        """获取缓存项"""
        async with self._lock:
            if key in self.cache:
                entry = self.cache[key]
                
                # 检查是否过期
                if entry.expires_at and datetime.now() > entry.expires_at:
                    await self._remove_entry(key)
                    return None
                
                # 更新访问信息
                entry.access_count += 1
                entry.last_accessed = datetime.now()
                
                # 移动到末尾（最近使用）
                self.cache.move_to_end(key)
                
                return entry
            
            return None
    
    async def set(self, key: str, value: Any, ttl_seconds: Optional[int] = None) -> bool:
        """设置缓存项"""
        async with self._lock:
            # 计算值的大小
            size_bytes = self._calculate_size(value)
            
            # 检查内存限制
            if size_bytes > self.max_memory_bytes:
                logger.warning(f"Cache entry too large: {size_bytes} bytes")
                return False
            
            # 确保有足够空间
            await self._ensure_space(size_bytes)
            
            # 创建缓存条目
            expires_at = None
            if ttl_seconds:
                expires_at = datetime.now() + timedelta(seconds=ttl_seconds)
            
            entry = CacheEntry(
                key=key,
                value=value,
                created_at=datetime.now(),
                expires_at=expires_at,
                size_bytes=size_bytes,
                cache_level=CacheLevel.MEMORY
            )
            
            # 如果key已存在，先移除旧的
            if key in self.cache:
                await self._remove_entry(key)
            
            # 添加新条目
            self.cache[key] = entry
            self.current_memory_bytes += size_bytes
            
            return True
    
    async def delete(self, key: str) -> bool:
        """删除缓存项"""
        async with self._lock:
            if key in self.cache:
                await self._remove_entry(key)
                return True
            return False
    
    async def clear(self):
        """清空缓存"""
        async with self._lock:
            self.cache.clear()
            self.current_memory_bytes = 0
    
    async def _remove_entry(self, key: str):
        """移除缓存条目"""
        if key in self.cache:
            entry = self.cache.pop(key)
            self.current_memory_bytes -= entry.size_bytes
    
    async def _ensure_space(self, required_bytes: int):
        """确保有足够的缓存空间"""
        # 检查数量限制
        while len(self.cache) >= self.max_size:
            oldest_key = next(iter(self.cache))
            await self._remove_entry(oldest_key)
        
        # 检查内存限制
        while (self.current_memory_bytes + required_bytes) > self.max_memory_bytes:
            if not self.cache:
                break
            oldest_key = next(iter(self.cache))
            await self._remove_entry(oldest_key)
    
    def _calculate_size(self, value: Any) -> int:
        """计算值的大小"""
        try:
            if isinstance(value, (str, bytes)):
                return len(value)
            elif isinstance(value, np.ndarray):
                return value.nbytes
            elif isinstance(value, dict):
                return len(json.dumps(value).encode('utf-8'))
            else:
                return len(pickle.dumps(value))
        except Exception:
            return 1024  # 默认1KB
    
    def get_stats(self) -> Dict[str, Any]:
        """获取缓存统计"""
        return {
            'size': len(self.cache),
            'max_size': self.max_size,
            'memory_usage_bytes': self.current_memory_bytes,
            'memory_usage_mb': self.current_memory_bytes / (1024 * 1024),
            'max_memory_mb': self.max_memory_bytes / (1024 * 1024),
            'memory_usage_percent': (self.current_memory_bytes / self.max_memory_bytes) * 100
        }


class FrameBuffer:
    """视频帧缓冲区 - 专门用于视频帧的循环缓冲"""
    
    def __init__(self, max_frames_per_camera: int = 100, max_cameras: int = 500):
        self.max_frames_per_camera = max_frames_per_camera
        self.max_cameras = max_cameras
        self.buffers: Dict[str, deque] = {}  # camera_id -> frame_deque
        self.frame_metadata: Dict[str, Dict[str, Any]] = {}  # frame_id -> metadata
        self._lock = asyncio.Lock()
    
    async def add_frame(self, camera_id: str, frame: np.ndarray, metadata: Dict[str, Any] = None) -> str:
        """添加视频帧"""
        async with self._lock:
            if camera_id not in self.buffers:
                if len(self.buffers) >= self.max_cameras:
                    # 移除最旧的摄像头缓冲区
                    oldest_camera = next(iter(self.buffers))
                    await self._remove_camera_buffer(oldest_camera)
                
                self.buffers[camera_id] = deque(maxlen=self.max_frames_per_camera)
            
            # 生成帧ID
            frame_id = f"{camera_id}_{int(time.time() * 1000)}_{len(self.buffers[camera_id])}"
            
            # 添加帧到缓冲区
            frame_entry = {
                'frame_id': frame_id,
                'frame': frame,
                'timestamp': datetime.now(),
                'camera_id': camera_id
            }
            
            self.buffers[camera_id].append(frame_entry)
            
            # 保存元数据
            if metadata:
                self.frame_metadata[frame_id] = metadata
            
            return frame_id
    
    async def get_frame(self, frame_id: str) -> Optional[Tuple[np.ndarray, Dict[str, Any]]]:
        """获取指定帧"""
        async with self._lock:
            # 从frame_id解析camera_id
            parts = frame_id.split('_')
            if len(parts) < 3:
                return None
            
            camera_id = '_'.join(parts[:-2])
            
            if camera_id not in self.buffers:
                return None
            
            # 查找帧
            for frame_entry in self.buffers[camera_id]:
                if frame_entry['frame_id'] == frame_id:
                    metadata = self.frame_metadata.get(frame_id, {})
                    return frame_entry['frame'], metadata
            
            return None
    
    async def get_latest_frame(self, camera_id: str) -> Optional[Tuple[str, np.ndarray, Dict[str, Any]]]:
        """获取摄像头的最新帧"""
        async with self._lock:
            if camera_id not in self.buffers or not self.buffers[camera_id]:
                return None
            
            latest_entry = self.buffers[camera_id][-1]
            frame_id = latest_entry['frame_id']
            frame = latest_entry['frame']
            metadata = self.frame_metadata.get(frame_id, {})
            
            return frame_id, frame, metadata
    
    async def get_frame_history(self, camera_id: str, count: int = 10) -> List[Tuple[str, np.ndarray, Dict[str, Any]]]:
        """获取摄像头的历史帧"""
        async with self._lock:
            if camera_id not in self.buffers:
                return []
            
            frames = []
            buffer = self.buffers[camera_id]
            
            # 获取最近的count个帧
            for frame_entry in list(buffer)[-count:]:
                frame_id = frame_entry['frame_id']
                frame = frame_entry['frame']
                metadata = self.frame_metadata.get(frame_id, {})
                frames.append((frame_id, frame, metadata))
            
            return frames
    
    async def remove_camera(self, camera_id: str) -> bool:
        """移除摄像头的所有帧"""
        async with self._lock:
            return await self._remove_camera_buffer(camera_id)
    
    async def _remove_camera_buffer(self, camera_id: str) -> bool:
        """移除摄像头缓冲区"""
        if camera_id in self.buffers:
            # 清理相关的元数据
            buffer = self.buffers[camera_id]
            for frame_entry in buffer:
                frame_id = frame_entry['frame_id']
                self.frame_metadata.pop(frame_id, None)
            
            # 移除缓冲区
            del self.buffers[camera_id]
            return True
        
        return False
    
    def get_stats(self) -> Dict[str, Any]:
        """获取缓冲区统计"""
        total_frames = sum(len(buffer) for buffer in self.buffers.values())
        
        return {
            'active_cameras': len(self.buffers),
            'max_cameras': self.max_cameras,
            'total_frames': total_frames,
            'max_frames_per_camera': self.max_frames_per_camera,
            'memory_usage_estimate_mb': total_frames * 2,  # 估算每帧2MB
            'cameras': {
                camera_id: len(buffer) 
                for camera_id, buffer in self.buffers.items()
            }
        }


class MultiLevelCache:
    """多级缓存系统"""
    
    def __init__(self):
        # 内存缓存
        self.memory_cache = LRUCache(
            max_size=settings.MAX_CONCURRENT_STREAMS * 10,
            max_memory_mb=512
        )
        
        # 帧缓冲区
        self.frame_buffer = FrameBuffer(
            max_frames_per_camera=settings.FRAME_BUFFER_SIZE,
            max_cameras=settings.MAX_CAMERAS
        )
        
        # Redis缓存
        self.redis_client: Optional[redis.Redis] = None
        self.redis_available = False
        
        # 缓存统计
        self.stats = CacheStats()
        
        # 配置
        self.default_ttl = settings.CACHE_TTL
        self.redis_ttl = settings.CACHE_TTL * 2  # Redis缓存时间更长
        
        # 监控任务
        self._monitoring_task = None
        self._cleanup_task = None
    
    async def initialize(self):
        """初始化多级缓存系统"""
        try:
            # 初始化Redis连接
            if REDIS_AVAILABLE:
                await self._init_redis()
            
            # 启动监控任务
            self._monitoring_task = asyncio.create_task(self._monitoring_loop())
            self._cleanup_task = asyncio.create_task(self._cleanup_loop())
            
            logger.info(f"Multi-level cache initialized. Redis available: {self.redis_available}")
            
        except Exception as e:
            logger.error(f"Failed to initialize multi-level cache: {e}")
            raise
    
    async def _init_redis(self):
        """初始化Redis连接"""
        try:
            self.redis_client = redis.from_url(
                settings.REDIS_URL,
                encoding="utf-8",
                decode_responses=False  # 保持二进制数据
            )
            
            # 测试连接
            await self.redis_client.ping()
            self.redis_available = True
            
            logger.info("Redis cache initialized successfully")
            
        except Exception as e:
            logger.warning(f"Redis initialization failed: {e}")
            self.redis_available = False
    
    async def get(self, key: str, cache_levels: List[CacheLevel] = None) -> Optional[Any]:
        """从多级缓存获取数据"""
        start_time = time.time()
        
        try:
            self.stats.total_requests += 1
            
            if cache_levels is None:
                cache_levels = [CacheLevel.MEMORY, CacheLevel.REDIS]
            
            # 按优先级查找缓存
            for level in cache_levels:
                value = await self._get_from_level(key, level)
                if value is not None:
                    self.stats.cache_hits += 1
                    
                    # 更新统计
                    if level == CacheLevel.MEMORY:
                        self.stats.memory_hits += 1
                    elif level == CacheLevel.REDIS:
                        self.stats.redis_hits += 1
                    elif level == CacheLevel.FRAME_BUFFER:
                        self.stats.frame_buffer_hits += 1
                    
                    # 如果从低级缓存获取，提升到高级缓存
                    if level != CacheLevel.MEMORY:
                        await self._promote_to_memory(key, value)
                    
                    return value
            
            # 缓存未命中
            self.stats.cache_misses += 1
            return None
            
        finally:
            # 更新平均访问时间
            access_time_ms = (time.time() - start_time) * 1000
            self._update_avg_access_time(access_time_ms)
    
    async def set(self, key: str, value: Any, ttl_seconds: Optional[int] = None, 
                  cache_levels: List[CacheLevel] = None) -> bool:
        """设置多级缓存数据"""
        try:
            if ttl_seconds is None:
                ttl_seconds = self.default_ttl
            
            if cache_levels is None:
                cache_levels = [CacheLevel.MEMORY, CacheLevel.REDIS]
            
            success = True
            
            # 设置到指定的缓存级别
            for level in cache_levels:
                level_success = await self._set_to_level(key, value, ttl_seconds, level)
                success = success and level_success
            
            return success
            
        except Exception as e:
            logger.error(f"Failed to set cache key {key}: {e}")
            return False
    
    async def delete(self, key: str, cache_levels: List[CacheLevel] = None) -> bool:
        """从多级缓存删除数据"""
        try:
            if cache_levels is None:
                cache_levels = [CacheLevel.MEMORY, CacheLevel.REDIS]
            
            success = True
            
            for level in cache_levels:
                level_success = await self._delete_from_level(key, level)
                success = success and level_success
            
            return success
            
        except Exception as e:
            logger.error(f"Failed to delete cache key {key}: {e}")
            return False
    
    async def _get_from_level(self, key: str, level: CacheLevel) -> Optional[Any]:
        """从指定缓存级别获取数据"""
        try:
            if level == CacheLevel.MEMORY:
                entry = await self.memory_cache.get(key)
                return entry.value if entry else None
            
            elif level == CacheLevel.REDIS and self.redis_available:
                data = await self.redis_client.get(key)
                if data:
                    return pickle.loads(data)
                return None
            
            elif level == CacheLevel.FRAME_BUFFER:
                # 帧缓冲区需要特殊处理
                return await self._get_from_frame_buffer(key)
            
            return None
            
        except Exception as e:
            logger.error(f"Failed to get from {level.value} cache: {e}")
            return None
    
    async def _set_to_level(self, key: str, value: Any, ttl_seconds: int, level: CacheLevel) -> bool:
        """设置到指定缓存级别"""
        try:
            if level == CacheLevel.MEMORY:
                return await self.memory_cache.set(key, value, ttl_seconds)
            
            elif level == CacheLevel.REDIS and self.redis_available:
                data = pickle.dumps(value)
                await self.redis_client.setex(key, self.redis_ttl, data)
                return True
            
            elif level == CacheLevel.FRAME_BUFFER:
                # 帧缓冲区需要特殊处理
                return await self._set_to_frame_buffer(key, value)
            
            return False
            
        except Exception as e:
            logger.error(f"Failed to set to {level.value} cache: {e}")
            return False
    
    async def _delete_from_level(self, key: str, level: CacheLevel) -> bool:
        """从指定缓存级别删除数据"""
        try:
            if level == CacheLevel.MEMORY:
                return await self.memory_cache.delete(key)
            
            elif level == CacheLevel.REDIS and self.redis_available:
                result = await self.redis_client.delete(key)
                return result > 0
            
            elif level == CacheLevel.FRAME_BUFFER:
                # 帧缓冲区删除逻辑
                return await self._delete_from_frame_buffer(key)
            
            return False
            
        except Exception as e:
            logger.error(f"Failed to delete from {level.value} cache: {e}")
            return False
    
    async def _promote_to_memory(self, key: str, value: Any):
        """将数据提升到内存缓存"""
        try:
            await self.memory_cache.set(key, value, self.default_ttl)
        except Exception as e:
            logger.error(f"Failed to promote key {key} to memory cache: {e}")
    
    async def _get_from_frame_buffer(self, key: str) -> Optional[Any]:
        """从帧缓冲区获取数据"""
        # 假设key格式为 "frame:{frame_id}" 或 "latest_frame:{camera_id}"
        if key.startswith("frame:"):
            frame_id = key[6:]  # 移除 "frame:" 前缀
            result = await self.frame_buffer.get_frame(frame_id)
            return result[0] if result else None  # 只返回帧数据
        
        elif key.startswith("latest_frame:"):
            camera_id = key[13:]  # 移除 "latest_frame:" 前缀
            result = await self.frame_buffer.get_latest_frame(camera_id)
            return result[1] if result else None  # 只返回帧数据
        
        return None
    
    async def _set_to_frame_buffer(self, key: str, value: Any) -> bool:
        """设置到帧缓冲区"""
        # 只有numpy数组（视频帧）才能设置到帧缓冲区
        if not isinstance(value, np.ndarray):
            return False
        
        if key.startswith("frame:"):
            # 从key中解析camera_id
            parts = key.split(':')
            if len(parts) >= 2:
                camera_id = parts[1].split('_')[0]  # 假设frame_id格式为 camera_id_timestamp_index
                await self.frame_buffer.add_frame(camera_id, value)
                return True
        
        return False
    
    async def _delete_from_frame_buffer(self, key: str) -> bool:
        """从帧缓冲区删除数据"""
        if key.startswith("camera:"):
            camera_id = key[7:]  # 移除 "camera:" 前缀
            return await self.frame_buffer.remove_camera(camera_id)
        
        return False
    
    def _update_avg_access_time(self, access_time_ms: float):
        """更新平均访问时间"""
        if self.stats.total_requests == 1:
            self.stats.avg_access_time_ms = access_time_ms
        else:
            # 使用指数移动平均
            alpha = 0.1
            self.stats.avg_access_time_ms = (
                alpha * access_time_ms + 
                (1 - alpha) * self.stats.avg_access_time_ms
            )
    
    # 视频帧专用方法
    
    async def add_video_frame(self, camera_id: str, frame: np.ndarray, metadata: Dict[str, Any] = None) -> str:
        """添加视频帧到缓冲区"""
        return await self.frame_buffer.add_frame(camera_id, frame, metadata)
    
    async def get_latest_video_frame(self, camera_id: str) -> Optional[Tuple[str, np.ndarray, Dict[str, Any]]]:
        """获取最新视频帧"""
        return await self.frame_buffer.get_latest_frame(camera_id)
    
    async def get_video_frame_history(self, camera_id: str, count: int = 10) -> List[Tuple[str, np.ndarray, Dict[str, Any]]]:
        """获取视频帧历史"""
        return await self.frame_buffer.get_frame_history(camera_id, count)
    
    # 监控和统计方法
    
    async def get_cache_stats(self) -> Dict[str, Any]:
        """获取缓存统计信息"""
        memory_stats = self.memory_cache.get_stats()
        frame_buffer_stats = self.frame_buffer.get_stats()
        
        redis_stats = {}
        if self.redis_available:
            try:
                info = await self.redis_client.info('memory')
                redis_stats = {
                    'used_memory': info.get('used_memory', 0),
                    'used_memory_human': info.get('used_memory_human', '0B'),
                    'connected_clients': info.get('connected_clients', 0)
                }
            except Exception as e:
                logger.error(f"Failed to get Redis stats: {e}")
        
        return {
            'overall': asdict(self.stats),
            'memory_cache': memory_stats,
            'frame_buffer': frame_buffer_stats,
            'redis_cache': redis_stats,
            'redis_available': self.redis_available
        }
    
    async def optimize_cache(self) -> Dict[str, Any]:
        """缓存优化 - 基于测试结果的增强版本"""
        optimization_results = {
            'memory_cache_cleared': 0,
            'redis_keys_cleaned': 0,
            'frame_buffers_optimized': 0,
            'optimization_actions': [],
            'performance_improvement': 0.0,
            'cache_stats_before': {},
            'cache_stats_after': {}
        }
        
        try:
            # 记录优化前的缓存状态
            stats_before = await self.get_cache_stats()
            optimization_results['cache_stats_before'] = stats_before
            
            # 内存缓存优化
            memory_stats = self.memory_cache.get_stats()
            memory_usage_percent = memory_stats['memory_usage_percent']
            
            if memory_usage_percent > 85:
                # 激进清理策略
                old_size = memory_stats['size']
                
                # 清理访问次数少的条目
                cleared_count = await self._optimize_memory_cache_aggressive()
                optimization_results['memory_cache_cleared'] = cleared_count
                
                optimization_results['optimization_actions'].append({
                    'action': 'aggressive_memory_cleanup',
                    'reason': f'内存使用率过高: {memory_usage_percent:.1f}%',
                    'items_cleared': cleared_count,
                    'memory_freed_mb': cleared_count * 2  # 估算每项2MB
                })
                
            elif memory_usage_percent > 70:
                # 温和清理策略
                cleared_count = await self._optimize_memory_cache_gentle()
                optimization_results['memory_cache_cleared'] = cleared_count
                
                optimization_results['optimization_actions'].append({
                    'action': 'gentle_memory_cleanup',
                    'reason': f'内存使用率较高: {memory_usage_percent:.1f}%',
                    'items_cleared': cleared_count
                })
            
            # Redis缓存优化
            if self.redis_available:
                try:
                    # 获取Redis内存使用情况
                    info = await self.redis_client.info('memory')
                    used_memory = info.get('used_memory', 0)
                    max_memory = info.get('maxmemory', 0)
                    
                    if max_memory > 0:
                        redis_usage_percent = (used_memory / max_memory) * 100
                        
                        if redis_usage_percent > 80:
                            # 清理过期键和低频访问键
                            keys_cleaned = await self._optimize_redis_cache()
                            optimization_results['redis_keys_cleaned'] = keys_cleaned
                            
                            optimization_results['optimization_actions'].append({
                                'action': 'redis_cleanup',
                                'reason': f'Redis内存使用率过高: {redis_usage_percent:.1f}%',
                                'keys_cleaned': keys_cleaned
                            })
                    
                    # 分析Redis键空间分布
                    keyspace_info = await self.redis_client.info('keyspace')
                    logger.info(f"Redis keyspace optimization completed: {keyspace_info}")
                    
                except Exception as e:
                    logger.error(f"Redis optimization failed: {e}")
            
            # 帧缓冲区智能优化
            frame_stats = self.frame_buffer.get_stats()
            active_cameras = frame_stats['active_cameras']
            max_cameras = frame_stats['max_cameras']
            
            if active_cameras > max_cameras * 0.9:
                # 缓冲区接近满载，需要优化
                optimized_cameras = await self._optimize_frame_buffers()
                optimization_results['frame_buffers_optimized'] = optimized_cameras
                
                optimization_results['optimization_actions'].append({
                    'action': 'frame_buffer_optimization',
                    'reason': f'帧缓冲区使用率过高: {(active_cameras/max_cameras)*100:.1f}%',
                    'cameras_optimized': optimized_cameras
                })
            
            # 缓存命中率优化
            overall_stats = stats_before.get('overall', {})
            hit_rate = overall_stats.get('hit_rate', 0)
            
            if hit_rate < 0.7:  # 命中率低于70%
                # 分析缓存模式并优化
                cache_pattern_optimization = await self._optimize_cache_patterns()
                
                optimization_results['optimization_actions'].append({
                    'action': 'cache_pattern_optimization',
                    'reason': f'缓存命中率过低: {hit_rate:.2%}',
                    'optimizations': cache_pattern_optimization
                })
            
            # 缓存预热策略
            if len(optimization_results['optimization_actions']) > 0:
                # 执行缓存预热
                prewarmed_items = await self._execute_cache_prewarming()
                
                optimization_results['optimization_actions'].append({
                    'action': 'cache_prewarming',
                    'reason': '优化后执行缓存预热',
                    'prewarmed_items': prewarmed_items
                })
            
            # 计算性能改善
            performance_score = 0
            for action in optimization_results['optimization_actions']:
                if action['action'] == 'aggressive_memory_cleanup':
                    performance_score += 20
                elif action['action'] == 'redis_cleanup':
                    performance_score += 15
                elif action['action'] == 'frame_buffer_optimization':
                    performance_score += 10
                elif action['action'] == 'cache_pattern_optimization':
                    performance_score += 25
                else:
                    performance_score += 5
            
            optimization_results['performance_improvement'] = min(50, performance_score)
            
            # 记录优化后的缓存状态
            stats_after = await self.get_cache_stats()
            optimization_results['cache_stats_after'] = stats_after
            
            logger.info(f"Cache optimization completed: {len(optimization_results['optimization_actions'])} actions taken, "
                       f"performance improvement: {optimization_results['performance_improvement']}%")
            
            return optimization_results
            
        except Exception as e:
            logger.error(f"Cache optimization failed: {e}")
            optimization_results['error'] = str(e)
            return optimization_results
    
    async def _optimize_memory_cache_aggressive(self) -> int:
        """激进的内存缓存优化"""
        try:
            cleared_count = 0
            current_time = datetime.now()
            
            # 获取所有缓存条目并按访问模式排序
            cache_items = list(self.memory_cache.cache.items())
            
            # 清理策略：优先清理访问次数少且时间久的条目
            items_to_remove = []
            
            for key, entry in cache_items:
                # 计算条目的价值分数
                age_hours = (current_time - entry.created_at).total_seconds() / 3600
                access_frequency = entry.access_count / max(age_hours, 0.1)
                
                # 低价值条目：访问频率低且占用空间大
                if access_frequency < 0.5 or entry.size_bytes > 10 * 1024 * 1024:  # 10MB以上
                    items_to_remove.append(key)
            
            # 清理低价值条目
            for key in items_to_remove[:len(cache_items) // 3]:  # 最多清理1/3
                await self.memory_cache.delete(key)
                cleared_count += 1
            
            return cleared_count
            
        except Exception as e:
            logger.error(f"Aggressive memory cache optimization failed: {e}")
            return 0
    
    async def _optimize_memory_cache_gentle(self) -> int:
        """温和的内存缓存优化"""
        try:
            cleared_count = 0
            current_time = datetime.now()
            
            # 只清理明显过期或很久未访问的条目
            cache_items = list(self.memory_cache.cache.items())
            
            for key, entry in cache_items:
                # 清理超过1小时未访问的大文件
                if entry.last_accessed:
                    hours_since_access = (current_time - entry.last_accessed).total_seconds() / 3600
                    if hours_since_access > 1 and entry.size_bytes > 5 * 1024 * 1024:  # 5MB以上
                        await self.memory_cache.delete(key)
                        cleared_count += 1
                        
                        if cleared_count >= 10:  # 温和清理，最多10个
                            break
            
            return cleared_count
            
        except Exception as e:
            logger.error(f"Gentle memory cache optimization failed: {e}")
            return 0
    
    async def _optimize_redis_cache(self) -> int:
        """优化Redis缓存"""
        try:
            if not self.redis_available:
                return 0
            
            keys_cleaned = 0
            
            # 扫描并清理过期键
            cursor = 0
            while True:
                cursor, keys = await self.redis_client.scan(cursor, count=100)
                
                for key in keys:
                    try:
                        # 检查键的TTL
                        ttl = await self.redis_client.ttl(key)
                        if ttl == -1:  # 没有过期时间的键
                            # 为旧键设置过期时间
                            await self.redis_client.expire(key, self.redis_ttl)
                        elif ttl > 0 and ttl < 60:  # 即将过期的键
                            # 让其自然过期
                            pass
                    except Exception:
                        # 键可能已经被删除
                        pass
                
                keys_cleaned += len(keys)
                
                if cursor == 0 or keys_cleaned > 1000:  # 限制扫描数量
                    break
            
            return keys_cleaned
            
        except Exception as e:
            logger.error(f"Redis cache optimization failed: {e}")
            return 0
    
    async def _optimize_frame_buffers(self) -> int:
        """优化帧缓冲区"""
        try:
            optimized_count = 0
            current_time = datetime.now()
            
            # 找出不活跃的摄像头缓冲区
            inactive_cameras = []
            
            for camera_id, buffer in self.frame_buffer.buffers.items():
                if buffer:
                    # 检查最后一帧的时间
                    latest_frame = buffer[-1]
                    frame_age = (current_time - latest_frame['timestamp']).total_seconds()
                    
                    # 超过5分钟没有新帧的摄像头视为不活跃
                    if frame_age > 300:
                        inactive_cameras.append(camera_id)
            
            # 清理不活跃的摄像头缓冲区
            for camera_id in inactive_cameras[:10]:  # 最多清理10个
                await self.frame_buffer.remove_camera(camera_id)
                optimized_count += 1
            
            return optimized_count
            
        except Exception as e:
            logger.error(f"Frame buffer optimization failed: {e}")
            return 0
    
    async def _optimize_cache_patterns(self) -> Dict[str, Any]:
        """优化缓存模式"""
        try:
            optimizations = {
                'ttl_adjustments': 0,
                'prefetch_enabled': False,
                'compression_enabled': False
            }
            
            # 分析缓存访问模式
            stats = await self.get_cache_stats()
            overall_stats = stats.get('overall', {})
            
            # 如果Redis命中率低，增加TTL
            redis_hits = overall_stats.get('redis_hits', 0)
            total_requests = overall_stats.get('total_requests', 0)
            
            if total_requests > 0 and redis_hits / total_requests < 0.3:
                # Redis命中率低，延长TTL
                self.redis_ttl = min(self.redis_ttl * 1.5, 7200)  # 最多2小时
                optimizations['ttl_adjustments'] = 1
            
            # 启用预取策略
            if overall_stats.get('cache_misses', 0) > 100:
                optimizations['prefetch_enabled'] = True
            
            return optimizations
            
        except Exception as e:
            logger.error(f"Cache pattern optimization failed: {e}")
            return {}
    
    async def _execute_cache_prewarming(self) -> int:
        """执行缓存预热"""
        try:
            prewarmed_count = 0
            
            # 预热常用的配置数据
            common_keys = [
                "system_config",
                "camera_list",
                "ai_algorithms_config",
                "performance_thresholds"
            ]
            
            for key in common_keys:
                # 模拟预热数据
                await self.set(f"prewarmed_{key}", {"prewarmed": True, "timestamp": datetime.now().isoformat()})
                prewarmed_count += 1
            
            return prewarmed_count
            
        except Exception as e:
            logger.error(f"Cache prewarming failed: {e}")
            return 0
    
    async def _monitoring_loop(self):
        """缓存监控循环"""
        while True:
            try:
                # 每分钟记录一次缓存统计
                stats = await self.get_cache_stats()
                
                # 记录关键指标
                overall_stats = stats.get('overall', {})
                hit_rate = overall_stats.get('hit_rate', 0.0)
                memory_hits = overall_stats.get('memory_hits', 0)
                redis_hits = overall_stats.get('redis_hits', 0)
                frame_buffer_hits = overall_stats.get('frame_buffer_hits', 0)
                
                logger.info(f"Cache Stats - Hit Rate: {hit_rate:.2%}, "
                           f"Memory Hits: {memory_hits}, "
                           f"Redis Hits: {redis_hits}, "
                           f"Frame Buffer Hits: {frame_buffer_hits}")
                
                # 检查是否需要优化
                if hit_rate < 0.7:  # 命中率低于70%
                    logger.warning("Low cache hit rate detected, consider optimization")
                
                await asyncio.sleep(60)  # 每分钟监控一次
                
            except Exception as e:
                logger.error(f"Error in cache monitoring loop: {e}")
                await asyncio.sleep(60)
    
    async def _cleanup_loop(self):
        """缓存清理循环"""
        while True:
            try:
                # 每小时执行一次优化
                await self.optimize_cache()
                await asyncio.sleep(3600)  # 每小时清理一次
                
            except Exception as e:
                logger.error(f"Error in cache cleanup loop: {e}")
                await asyncio.sleep(3600)
    
    async def shutdown(self):
        """关闭多级缓存系统"""
        try:
            # 停止监控任务
            if self._monitoring_task:
                self._monitoring_task.cancel()
            if self._cleanup_task:
                self._cleanup_task.cancel()
            
            # 清理缓存
            await self.memory_cache.clear()
            
            # 关闭Redis连接
            if self.redis_client:
                await self.redis_client.close()
            
            logger.info("Multi-level cache shutdown completed")
            
        except Exception as e:
            logger.error(f"Error during cache shutdown: {e}")


# 全局多级缓存实例
multi_level_cache = MultiLevelCache()